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Section  1 


INTRODUCTION 

This  study  was  performed  in  support  of  the  Manned  Threat  Quantification 
(MTQ)  program  at  AFAMRL.  This  program  is  concerned  with  modeling  the  human 
operator  as  part  of  surface-to-air  weapon  systems.  Initial  work  focused  on 
the  human  operator  as  a  target  tracker  for  antiaircraft  artilJery  (AAA) 
systems.  This  work  was  extended  to  surface-to-air  missile  systems  (SAM)  in 
the  study  documented  in  this  report.  Subsequent  experimentation  will  be 
concerned  with  the  human  operator  as  a  decision  maker  and  strategist. 

The  two  main  objectives  for  conducting  this  study  were  to: 

(1)  Provide  a  data  base  for  developing  a  model  of  the  human  operator 
in  the  manual  TV  tracking  mode  of  a  SAM  system. 

(2)  Obtain  data  which  would  assist  in  structuring  additional  MTQ  SAM 
simulation  experiments;  future  experiments  will  emphasize  decision 
making  processes  by  the  human  operators. 
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Section  2 


EXPERIMENTAL  PLAN 


A.  FACILITIES  AND  APPARATUS 

The  facilities  and  apparatus  for  this  study  were  located  at  the  Air  Force 
Aerospace  Medical  Research  Laboratory,  Building  33,  Wr ight-Patterson  Air 
Force  Base,  Ohio.  Figures  1  and  2  depict  the  SAM  system  and  the  simulation. 
The  purpose  of  the  simulation  was  to  present  to  the  human  operators  the  TV 
image  and  control  functions  as  in  a  SAM  system  during  a  combat  engagement. 
Hence,  the  target  appeared  to  move  like  a  real  aircraft,  and  the  simulated 
tracking  sight  appeared  to  move  with  the  same  dynamics  as  a  real  sight. 

1 .  Consoles 

The  consoles  used  by  the  elevation  and  azimuth  operators  are  shown  in 
Figures  3  and  A,  respectively.  The  TV  displays  presented  identical  infor¬ 
mation  to  both  operators  via  Conrac  RQB-14  black  and  white  monitors.  These 
monitors  were  oriented  to  produce  a  vertical  scan  which  is  like  many  SAM 
system  TV  displays.  Horizontal  and  vertical  lines  were  present  on  the 
displays  as  shown  in  Figure  5.  The  black  vertical  lines  could  be  used  to 
judge  launch  boundaries  for  the  missiles  by  comparing  line  spacing  with  the 
wingspan  of  the  target;  this  was  not  part  of  this  study.  Subjects  were 
instructed  to  use  the  crosshairs  as  a  reference  and  to  keep  the  target 
centered  at  all  times. 

The  "perfect  track"  switches  were  used  primarily  during  training  to  bypass 
the  inputs  from  the  tracking  operator's  crank  and  keep  the  target  centered 
tor  that  axis.  This  helped  subjects  to  reacquire  the  target  if  it  left  the 
field  of  view.  These  switches  were  also  used  during  daily  calibrations. 

The  "breaklock/reacquire"  switch  was  used  to  put  a  signal  onto  tlte  data  tape 
so  that  subjects  could  indicate  when  the  target  had  left  the  field  of  view 
and  if  they  had  reacquired  the  target.  The  "run/stop"  switch  was  used  to 
initiate  the  training  and  data  sessions  or  to  stop  a  session  prematurely,  if 
needed.  "Breaklock/reacquire"  and  "run/stop"  switches  were  not  present  on 
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Figure  1.  Representative  SAM  Tracking  System 
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Figure  5.  TV  Display  Lines 
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the  elevation  operator's  console  since  a  need  for  duplicate  controls  was 
never  demonstrated.  The  handwheels  allowed  the  trackers  to  make  rate  con¬ 
trol  inputs  in  order  to  keep  the  target  centered  for  their  respective  axis, 
azimuth,  or  elevation. 

2 .  Video 

The  video  image  presented  to  the  trackers  was  obtained  by  mixing  an  aircraft 
image  with  a  terrain  background  image.  The  aircraft  image  was  generated  by 
having  a  TV  camera  view  a  scale  model  of  an  F-4  aircraft  that  was  moved  in 
roll,  pitch,  and  yaw  by  computer  command  so  that  the  aircraft's  orientation 
appeared  the  same  way  it  would  to  the  trackers  in  a  real  SAM  system. 

Tracking  error  between  the  aircraft  and  the  boresight  was  used  to  generate 
commands  for  the  deflection  circuits  of  the  TV  camera  so  that  the  target 
appeared  to  move  up /down  and  left /right.  The  size  of  the  aircraft  image  was 
determined  by  the  model  size,  path  length  between  the  model  and  camera,  and 
the  camera's  zoom  lens.  The  computer  used  the  target's  slant  range,  con¬ 
tained  on  the  trajectory  disc,  to  control  the  zoom  lens.  The  size  of  the 
aircraft  image  equalled  the  apparent  size  of  the  aircraft  as  a  function  of 
slant  range  when  viewed  through  10X  optics.  For  example,  at  a  simulated 
slant  range  at  4  km,  the  wingspan  of  the  target  would  be  2.5  cm  wide  which 
is  equivalent  to  4  mrad.  The  field  of  view  observed  by  the  trackers  was 
1.5  degrees  in  azimuth  and  1.0  degree  in  elevation.  The  aircraft  imago  was 
darker  than  the  background.  The  background  was  generated  by  having  another 
camera  view  a  silhouette  (see  Figure  b)  which  was  drawn  onto  the  acetate 
film.  This  film  moved  in  left/right  and  up/down  motions  under  computer 
control  so  that  the  TV  image  appeared  to  move  just  as  it  would  when  the  SAM 
system's  TV  camera  was  moved  in  azimuth  and  elevation.  Although  the  back¬ 
ground  did  not  look  like  a  real  terrain  scene,  previous  research  (Crabtree 
and  Nelson,  1979)  indicated  that  it  would  have  no  adverse  affect  on  tracking 
performance  for  the  target  elevation  angles  and  atmospheric  conditions  used 
here . 
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Figure  6.  Silhouette  Background 


3. 


Plant  Dynamics 

The  plant  dynamics  for  the  control  system  are  shown  in  Figure  7,  and  the 
resolution  for  the  simulation  is  documented  in  Table  1.  The  differences  in 
resolution  between  azimuth  and  elevation  were  due  to  the  fixed  number  of 
"bits  of  resolution"  which  was  used  to  scale  a  large  dynamic  range  in 
azimuth  but  a  small  range  in  elevation. 

TABLE  1.  SIMUIATION  RESOLUTION 


Ax  is 

Resolution  Parameter 

Az imuth 

Elevation 

Target  Position 

.0959  mrad 

.0256  mrad 

Boresight  Position 

.1  mrad 

1  .0417  mrad 

"Data  to  Tape" 

.00129  mrad 

.00129  mrad 

Total 

.19719  mrad 

j  .06859  mrad 

! _ 

B.  INDEPENDENT  VARIABLE 

The  independent  variable  in  this  study  was  "trajectory."  In  order  to  sample 
the  human  operator's  performance,  it  was  decided  to  select  various  target 
trajectories  wh ich  were  thought  to  affect  the  tracker's  behavior.  These 
trajectories  are  listed  in  Table  2  and  depicted  graphically  in  Figure  8  and 
Appendix  A.  Trajectories  l,  2,  and  5  were  selected  to  provide  a  range  of 
azimuth  rates  from  low  to  high.  High  was  defined  as  a  value  which  approxi¬ 
mated  the  maximum  steady  state  rate  of  the  servo.  Trajectories  3  and  4  were 
used  to  investigate  the  effects  of  nonlinear  trajectories. 

C.  DEPENDENT  VARIABLES 

The  dependent  variables  were  (l)  azimuth  error,  (2)  elevation  error,  and 
(i)  combined  angular  error.  The  first  two  were  defined  as  the  angular  dif¬ 
ference,  in  azimuth  or  elevation,  respectively,  between  the  target  and  the 
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el  error 


Figure  7.  Plant  Dynamics 
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boresight  of  the  simulated  TV  tracker.  Combined  angular  error  was  obtained 
by  using  the  formula: 

Combined  Angular  Error  =  i - 

\  (azimuth  error)  +  (elevation  error) 

Each  of  the  metrics  was  computed  every  25  msec  during  data  collection,  and 
the  data  recorded  on  magnetic  tape. 

The  dependent  variables  (azimuth,  elevation,  and  combined  tracking  errors) 

were  selected  on  the  basis  of  providing  data  about  the  human  operator's 

individual  and  team  performance.  The  azimuth  and  elevation  tracking  errors 

are  useful  for  development  of  the  human  operator  models,  and  the  combined 

tracking  error  is  a  useful  metric  of  total  tracking  performance.  It  relates 

how  well  the  boresight  was  kept  on  the  target,  independent  of  individual 

axis  error.  Eor  a  more  complete  view  of  system  perf ormance ,  these  tracking 

errors  must  he  entered  into  an  attrition  model  such  as  TAG  ZINGER  so  that 

one  can  determine  missile  miss  distance  and  P  .  The  angular  error  provides 

K 

only  limited  information  since  it  does  not  consider  slant  range,  missile 
trajectory,  missile  warhead  characteristics,  or  any  other  factors  normally 
included  in  an  engagement  analysis. 

Summary  statistics,  for  data  analysis,  were  computed  for  the  above  metrics 
and  consisted  of  RMS  error  as  shown  below: 


Ibis  was  calculated  for  azimuth  error,  elevation  error,  and  combined  angular 
error.  Those  statistics  were  selected  since  they  have  been  shown  to  be 
reasonably  reliable  and  sensitive  measures  of  human  operator  performance  in 
previous  studies. 


lb 


D. 


SUBJECTS 


Subjects  consisted  of  male  and  female  college  students  in  the  18  to  22  age 
group.  Their  vision  was  20-20  or  corrected  to  that  value.  Subjects  had 
previous  experience  with  tracking  simulators  including  the  one  used  in  this 
study . 

Only  two  teams.  Team  Number  1  and  Team  Number  4,  were  selected  from  the  pool 
of  available  subjects  because  of  the  needs  of  the  modeling  effort.  The 
models  typically  require  a  high  number  of  replications  in  order  to  obtain  a 
stable  variance  estimate  but  do  not  incorporate  large  numbers  of  subjects. 
Prior  to  participation  in  the  experiment,  subjects  were  required  to  complete 
a  consent  form  (see  Appendix  B)  . 

E.  DESIGN 


The  plan  for  the  training  sessions,  data  collection  sessions,  and  ordering 
of  treatments  (trajectories)  is  documented  in  Figure  9  and  Table  3. 

Ordering  of  the  treatments  was  done  with  an  awareness  for  (1)  subjects' 
ability  to  "guess"  the  treatment  and  (2)  sequence  effects.  However,  within 
a  few  seconds  after  the  onset  of  the  trajectory,  subjects  knew  which  trajec¬ 
tory  was  being  administered.  Session  length  was  limited  to  approximately 
three  quarters  of  an  hour  in  an  attempt  to  reduce  boredom  and  fatigue  for 
the  subjects. 

F.  PROCEDURES 


During  training,  subjects  were  instructed  to  minimize  the  error  between  the 
target  and  the  crosshairs  for  their  respective  axis  by  using  their  handwheel. 
To  help  the  subjects  assess  how  well  they  were  tracking,  a  feedback  score 
was  computed  and  displayed  on  an  auxiliary  CRT  after  each  trajectory  was 
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EXPERIMENT 


TABLE  3.  EXPERIMENTAL  DESIGN  -  TREATMENTS 
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Note:  Team  1  stopped  with  replication  16,  Team  4  stopped  with 

replication  20. 


comp  1 e tod .  The  feedback  metric  was  computed  separately  for  each  axis  and 
consisted  ot  computing  the  average  of  the  absolute  angular  error 

(4 

based  on  a  sample  rate  of  2b  msec.  The  score  for  that  trajectory  was  then 
multiplied  by  1U0.  Multiplying  by  100  was  done  to  avoid  having  small  num¬ 
bers  as  a  feedback  score.  Subjects  in  previous  studies  indicated  that  they 
could  relate  to  a  scale  such  as  0  to  100  better  than  .00  to  1.00.  If  a 
subject  had  difficulty  tracking  a  target,  lie  could  use  the  perfect  track 
switch  to  keep  the  target  centered  so  that  the  target  wouldn't  leave  the 
field  of  view  and  thus  prevent  the  other  subject  from  tracking  in  his  axis. 

An  average  was  computed  for  the  combined  absolute  angular  error  and  was  used 
by  the  training  supervisor  and  principal  investigator  to  judge  when  the 
subjects  had  reached  an  acceptable  level  of  performance  for  all  of  the 
trajectories.  When  this  point  was  reached ,  data  collection  began. 

Procedures  during  data  collection  were  essentially  the  same  as  those  during 
training.  bach  day,  prior  to  data  collection,  calibration  procedures  were 
performed  to  insure  that  the  simulator  was  operating  properly  (see 
Appendix  C) .  Presorted  keypunch  cards  were  loaded  into  the  computer 
according  to  the  experimental  design.  At  the  beginning  of  each  session  was 
a  calibration  trajectory  during  which  subjects  were  not  tracking  and  the 
simulator  was  in  "perfect  track."  This  served  as  another  check  of  system 
operation.  Use  of  the  perfect  track  switch  during  data  collection  was  not 
allowed.  Subjects  tracked  each  target  trajectory  as  well  as  possible,  and 
observed  their  respective  feedback  scores  at  the  end  of  each  trial.  After 
20  trials,  the  next  team  began  tracking.  While  a  team  might  have  completed 
two  sessions  in  a  given  day,  there  was  always  a  rest  period  between  sessions. 
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RKSUI.TS 

Four  data  collection  sessions  were  completed  for  i'eam  1,  and  five  were 
completed  for  ream  4,  This  was  equivalent  to  lb  and  '20  replications  per 
trajectory  for  each  team,  respectively.  Due  to  hardware  reliability  prob¬ 
lems,  the  project  team  agreed  to  terminate  data  collection  at  this  point  for 
the  following  reasons: 

(1)  Team  performance  appeared  to  be  relatively  stable  and  completing 
the  design  was  not  expected  to  cause  a  significant  change  in  the 
tracking  error  metrics. 

(2)  it  was  impossible  to  estimate  future  delays.  Additional  delays 
would  be  serious  due  to  subject  availability  which  would  necessi¬ 
tate  (a)  discarding  the  current  data  base  and  (h)  retraining  new 
teams . 

Unless  otherwise  noted,  all  data  analysis  considered  only  the  first  4 
sessions  or  In  replications.  Session  S  was  deleted  lor  Team  4.  Approxima- 
t  ion  techniques  could  have  been  used  to  estimate  missing  data  for  Team  1; 
however,  because  of  the  small  number  of  subjects  in  the  study,  this  would 
not  have  been  .is  advisable  as  truncating  the  data  from  Team  4. 

A.  VISUAL  KXAMLNATION  OF  DATA 

Plots  of  the  average  absolute  combined  tracking  error  scores,  used  to  monitor 
team  performance  on  a  session-bv-session  basis,  are  shown  in  Figures  10  and 
11  for  Teams  1  and  4,  respectively.  These  plots  indicated: 

(1)  SLOW  5A  trajectory  was  tracked  witli  the  lowest  error,  KR  S-I’ASS 
and  FLYBY  ID  had  the  highest  error,  and  FLYBY  IB  and  F.K  ZKI-ZAU 
clustered  In  between. 


(2)  Team  4's  tracking  accuracy  was  slightly  better  than  Team  1  for 
FLYBY  ID. 

(3)  Tracking  errors  decreased  as  a  function  of  replication:  FLYBY  11) 
and  ER  S-PASS  showed  the  largest  change,  ER  XIC-/.AC  and  FLYBY  IB 
showed  moderate  change,  and  SLOW  5A  showed  the  least  change. 

Plots  were  generated  that  show  the  average  tracking  error  and  standard 
deviation  as  a  function  of  time  for  azimuth  and  elevation  lor  each  team  (see 
Appendix  D) .  Note  that  these  plots  were  based  on  lb  replications  of  each 
experimental  condition  for  Team  1  and  20  replications  for  Team  4.  A  visual 
inspection  of  these  plots  indicated: 

(1)  Tracking  errors  were  not  constant  during  the  trajectory;  larger 
errors  were  seen  in  elevation  than  azimuth  in  the  region  around 
crossover. 

(2)  Little  difference  was  observed  between  teams  for  mean  tracking 
errors  but  peaks  in  standard  deviation  were  higher  in  azimuth  for 
Team  1  and  higher  in  elevation  for  Team  4. 

B.  STATISTICAL  ANALYSIS  OF  EXPERIMENTAL  EFFECTS 

RMS  tracking  error  scores  were  computed  for  azimuth,  elevation,  and  combined 
errors  with  only  the  first  16  replications  used  for  analysis  purposes  as 
noted  earlier.  Examinations  of  the  raw  data  and  questioning  the  teams  indi¬ 
cated  that  there  were  no  instances  where  the  target  left  the  field  of  view 
(breaklocks)  and  no  instances  of  perfect  track  switch  activations.  Examina¬ 
tion  of  the  RMS  scores  for  skew  and  kurtosis  indicated  that  the  data  were 
approximately  normal,  and  there  was  no  need  for  outlier  elimination  or 
transformation  of  the  data  prior  to  the  statistical  analysis. 


Analysis  of  Overall  Effects 


1  . 

A  mixed  model  analysis  of  variance  (ANOVA)  was  performed  for  the  RMS 
azimuth,  RMS  elevation,  and  RMS  combined  tracking  scores.  The  fixed  effects 
were  (a)  trajectory  and  (b)  session  with  team  as  the  random  effect.  The 
results  are  presented  in  Tables  4,  5,  and  6.  Only  the  first  four  sessions 
were  used  in  this  analysis  in  order  to  use  the  same  number  of  replications 
tor  each  team.  This  analysis  was  used  to  test  the  hypotheses  of  no  team 
ditto  fences ,  no  trajectory  differences,  no  session  differences,  and  no 
i n t erne t ions .  Means  and  standard  deviations  for  each  cell  in  the  ANOVAs  are 
presented  in  Appendix  K.  Examination  of  the  relationship  of  the  cell  means 
and  standard  deviations  did  not  indicate  any  undesirable  correlation  which 
would  have  raised  questions  concerning  the  appropriateness  of  the  ANOVA. 


Analysis  of  Trajectory  Effects 


In  order  to  determine  which  trajectories  were  responsible  for  "trajectory" 
being  significant  in  the  ANOVAs,  Tukey  Honestly  Significant  Difference  (USD) 
tests  were  performed.  These  tests  were  applied  to  all  pairs  of  cell  means 
for  trajectories,  and  the  results  of  these  tests  are  shown  in  Appendix  (•’. 

I'h  is  test  consists  of  comparing  the  difference  between  two  means  with  a 
value  based  on  the  Student ized  Range  statistic  (Ouenther,  1 Dh4 ;  Keppel, 

ID  /  1 ;  Morr i son ,  I  Do  7 ) . 


A  general  summary  of  the  results  from  the  USD  tests  is  shown  in  Figure  12. 
Results  of  the  USD  tests  were  combined  witli  plots  of  RMS  tracking  error  vs. 
trajectory  and  are  presented  in  Figures  IT,  14,  and  IS. 


Analysis  of  Session  and  Replication  Effects 


"Session"  was  found  to  he  a  significant  effect  in  the  ANOVAs  for  the 
azimuth,  elevation,  and  combined  scores.  In  order  to  gain  some  insight  into 
tlu*  nature  ot  the  session  effect,  plots  of  mean  RMS  error  vs.  session  were 
prepared  (see  Figure  1<>).  As  can  he  seen  iron-  these  plots,  tracking  errors 
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TRAJECTORY 


o 


Figure  14.  RMS  Elevation  Scores  Versus  Trajectory  with  HS1)  Test  Results 


RMS  Combined  Scores  Versus  Trajectory  with  USD  Test  Results 


MEAN  RMS  ERROR  (MRAD) 


decreased  for  every  session  with  the  least  change  between  Sessions  1  and  2 . 
Examination  of  tracking  scores  vs.  replication,  rather  than  session,  gives  a 
more  detailed  picture  of  progressive  effects  (see  Appendix  G)  .  While  the 
overall  trend  is  more  difficult  to  perceive,  it  is  obvious  that  the  data 
were  not  stable  over  time.  Variations  were  not  extremely  large,  however. 

4 .  Analysis  of  Team  Effects 

Team  effects  were  found  to  be  statistically  significant  for  azimuth  and 
elevation  scores  (Figures  4  and  5)  but  not  for  combined  scores  (Figure  b). 
Additional  results  which  suggest  team  differences  can  be  found  in  Figures  10 
and  11  and  in  Appendices  D,  E,  and  G. 

5 .  interaction  Effects 

Statistical  significance  was  achieved  for  the  interaction  of  team  and 
trajectory  for  the  azimuth,  elevation,  and  combined  tracking  scores 
(Tables  4,  5,  and  6).  Further  analysis  was  not  warranted,  hut  the  issue  is 
treated  again  in  the  Discussion  section. 
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Sect  ion  4 
DISCUSSION 


A.  TRAJECTORIES 

1'he  trajectories  u  ed  in  this  study  must  be  considered  "familiar"  to  the 
subjects  due  to  training,  number  of  replications,  and  previous  experience. 
While  unfamiliar  trajectories  might  have  induced  higher  initial  tracking 
errors,  the  errors  would  likely  become  smaller  with  more  replications.  The 
decrease  in  RMS  error  across  session,  shown  in  Figure  15,  also  supports  this 
conclusion.  Requirements  of  the  human  operator  model  development,  however, 
created  a  need  for  a  large  number  of  replications.  In  spite  of  attempts  to 
use  difficult  trajectories,  i.e.,  high  angular  rates  or  maneuvering, 
tracking  errors  were  not  extremely  high.  They  were  less  than  5  mrad  maximum 
as  indicated  by  the  ensemble  average  tracking  error  plots  (Appendix  D)  with 
no  breakloeks. 

The  analysis  of  the  summary  statistics  and  examination  of  tracking  errors  as 
a  function  of  time  were  in  agreement  for  establishing  that  trajectory 
effects  existed.  The  ANOVAs  for  azimuth,  elevation,  and  combined  errors 
indicated  that  trajectory  differences  did  exist.  The  Tukey  USD  tests  sub¬ 
stantiated  this  finding  also  (reference  Figures  Id,  14,  and  15),  but  the 
trajectory  diifurences  were  not  extremely  strong.  From  Figure  12,  it  can  be 
seen  that  the  trajectories  fell  into  an  almost  uniform  ranking  for  all  three 
metrics.  Further  examination ,  however,  indicated  that  differences  between 
"adjacent"  trajectories  were  not  statistically  significant  except  for  three 
instances.  This  figure  bears  some  resemhlence  to  the  trajectory  character¬ 
istics  (Table  2)  as  noted  below: 

(1)  Trajectory  2,  SLOW  5A,  had  the  lowest  scores;  and  this  trajectory 
could  he  considered  the  easiest  since  it  was  a  1 1 vhy  w i rh  low 
angular  rates  for  both  axes. 


(2)  Trajectory  1,  FLYBY  IB,  would  be  expected  to  have  higher  azimuth 
tracking  errors  due  to  tie  higher  angular  rate  in  this  axis.  It 
is  unusual  that  there  wei e  significant  differences  between  Trajec¬ 
tories  1  and  2  for  elevation  but  not  for  azimuth.  This  could  be 
due  to  the  differences  between  individual  operators  (see  "B.  TLAMS/ 
SUBJECTS"  below) . 

(3)  Trajectory  4,  EK  ZIC-ZAG,  could  be  expected  to  yield  higher 
azimuth  tracking  errors  since  the  trajectory  is  not  as  predictable 
as  the  flybys,  Trajectories  1  and  2.  Azimuth  scores  for  trajec¬ 
tories  fell  into  an  ordinal  relationship  of  1,  4,  and  3;  but 
elevation  scores  fell  into  one  group  for  these  trajectories.  This 
ordinal  relationship  appears  reasonable  since  it  coincides  with 
the  "no  azimuth  perturbations,"  "slight  azimuth  oscillations,"  and 
"high  azimuth  rate  at  egress"  characteristics  of  these 

t  raj  ector ies  . 

(4)  Trajectories  3,  HR  S-PASS,  and  5,  FLYBY  11),  are  related  as  such 
due  possibly  to  the  "excessive"  azimuth  rate  of  Trajectory  3  and 
the  "elevation  maneuver"  of  Trajectory  3. 

Trajectory  effects  were  also  observed  when  visual  examinations  of  ensemble 
tracking  errors  vs.  traject  >ry  accelerations  were  made  for  each  respective 
axis  (see  Appendices  A  and  D) .  It  was  observed  that  as  azimuth  or  elevation 
angular  acceleration  increased,  the  ensemble  tracking  error  and  standard 
deviation  for  that  axis  also  increased.  This  typically  occurred  at  cross¬ 
over  or  during  a  maneuver.  Trajectory  2,  SLOW  5A,  was  an  exception  which 
yielded  relatively  uniform  performance  over  the  entire  trajectory.  Because 
this  trajectory  had  such  low  angular  rates  and  the  tracking  was  so  stable, 
it  could  be  considered  a  baseline  for  the  team's  best  possible  performance. 

B.  TEAMS/ SUBJECTS 

It  is  possible  that  the  data  would  have  fallen  into  more  uniform  patterns  if 
the  better  azimuth  operator  was  paired  with  the  better  elevation  operator. 

If  tills  was  done,  then  one  team  could  have  been  identified  as  "better" 


since  tin'  trucking  1'iTur.s  would  have  Ih'oii  lower  !or  both  axes.  With  the 
sul'  ii'i'ts  usoil  in  tliis  study,  "tin-  host"  team  would  consist  ol  Team  l's 
elevation  operator  and  Team  4's  ax  i mu  ill  operator.  The  other  subjects  would 
then  constitute  "the  second  best"  team. 

Hie  ensemble  average  tracking,  urmr  (Appendix  li)  were  given  a  visual 
inspection  to  see  it  different  strategies  might  have  existed.  Only  two 
;a i nor  differences  were  observed: 

(1)  For  Trajectory  1,  FLYBY  IB,  Team  l's  azimuth  error  alternated  from 
the  negative  region  to  the  positive  region  at  crossover,  but 

Team  4's  error  lacked  the  negative  region. 

(2)  For  Trajectory  2,  SLOW  5A,  Team  l's  azimuth  error  was  essentially 
in  the  positive  region  while  Team  4's  was  in  the  negative  region. 

Team  differences  were  significant  for  azimuth  and  elevation  scores  but  not 
for  the  combined  scores.  Team  4  showed  lower  error  scores  than  Team  1  for 
azimuth,  and  the  reverse  was  true  for  elevation.  This  reversal  between 
azimuth  and  elevation  errors  most  likelv  caused  the  lack  of  a  significant 
difference  between  teams  for  the  combined  score.  These  team  differences 
were  also  confirmed  in  the  plots  of  (1)  RMS  errors  vs.  replications 
(Appendix  IT)  and  (2)  ensemble  average  tracking  errors  (Appendix  D) . 

1 1 .  S  ITS. S  IONS  /  K  FPL  [CATIONS 

There  were  several  indications  that  effects  due  to  session  existed.  Average 
absolute  combined  tracking  errors  (Figures  10  and  11),  and  Mean  RMS  Krror 
(Figure  lb)  indicated  slight  improvements  with  each  session.  Session  was 
found  to  be  statistically  significant  for  azimuth,  elevation,  and  combined 
RMS  errors  based  on  the  ANOVAs.  Finally,  the  plots  of  RMS  error  scores  vs. 
replication  (Appendix  G)  also  showed  progressive  effects.  These  effects 
were  rel.it ively  small;  and  if  they  had  been  larger,  one  would  have  to  ques¬ 
tion  the  criterion  used  to  determine  that  the  subjects  were  adequately 
t  ru  i  m  il . 
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I). 


INTERACTIONS 


The  ANOVAs  Indicated  a  significant  interaction  between  team  and  trajectory 
for  the  azimuth,  elevation,  and  combined  scores.  The  apparent  cause  for 
this  significant  interaction  was  an  increase  in  disparity  between  teams  for 
the  more  difficult  trajectories.  Examination  ot  means  and  standard  devia¬ 
tions  (Appendix  E)  indicated  that  as  trajectories  become  more  difficult,  due 
to  higher  angular  rates  or  maneuvering,  the  difference  between  teams 
increased . 
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Section  S 


CONCLUSIONS 


A.  l'RAJECTORl  KS 

Diltcrcnci's  among  trajectories  were  observed  based  on  the  a/.iiimtli,  eleva¬ 
tion,  and  combined  tracking  scores.  The  tr.i  jectories  could  be  grouped  in 
the  order  shown  below: 


TABLE  7.  RANK  ORDERING  OE  TRAJECTORIES 


The  tracking  errors  appeared  to  be  related  to  the  angular  accelerations  of 
the  trajectories  with  an  increase  in  mean  error  and  standard  deviation  as 
angular  acceleration  increased. 

B.  TEAMS  OR  SUBJECTS 

Diltereuces  between  teams  were  found  for  azimuth  and  elevation  tracking 
scores  but  not  for  combined  scores. 

I'eam  ->  performed  better  for  minimizing  azimuth  tracking,  errors;  Team  1 
performed  better  tor  minimizing  elevation  tracking  errors.  Ensemble  aver 
ages  of  tracking,  error  ns  a  function  of  time  indicated  only  minor  strategy 
differences  between  teams. 
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S  t  a  t  i  s  t  i  ca  1  1  significant  d  i  t  I  e  rciii'cs  were  obtained  lor  ■.<•■,',1.111 
a;’,  i  r.111 1  li ,  elevation,  anil  combined  scurcs.  riu-  1  "..iiji  i  l  mie  ol  the:, 
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I).  1  NTKKAOT  IONS 


llii’ ro  was  a  statistically  significant  interaction  between  to.i::i 
trajectory  factors  for  azimuth,  elevation,  and  combined  scores, 
for  the  interaction  is  believed  to  be  less  divergence  between  t 
easv  trajectories  than  the  difficult  trajectories  (see  Appendix 


K.  CKNKKAl, 


Although  statistically  significant  dilferences  were  obtained,  ,1 
errors  were  relatively  small:  1  .(LSd  mra-1  maximum  RMS  error  and 
iilix  iiinin  ensemble  average  error.  Little  change  would  be  export! 
lull  -ill  replications  were  completed  since  within  re  p  I  i  ea  t  i  on  it 
small.  Die  impact  of  these  small  human  operator  tracking  on  or 
weapon  system  performance  (i.e.,  miss  distance  and  I' .  1  will  be 
Inline  MTi)  SAM  simulations  and  analyses.  Comparisons  will  he  1: 
human  operator  tracking,  and  radar  auto! rack. 
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Section  (1 


RECOMMENDATIONS 

Some  additional  work  reina  ins  to  he  done  to  quantify  the  characteristics  of 
the  human  operator  for  basic,  stable  tracking  performance.  Within  this 
context,  the  following  research  topics  are  recommended  for  subsequent  experi¬ 
ments  : 

(1)  Explore  the  upper  limits  of  "trajectory  difficulty,"  keeping  in 
mind  that  "breaklocks"  are  unlikely  to  occur  when  tracking  in  all 
manual  TV  and  radar  modes. 

(2)  Incorporate  trajectories  with  more  complex  maneuvers  to  cause 
variations  in  azimuth  and  elevation  tracking  performance 
simul taneously. 

(5)  Use  unfamiliar  trajectories  with  well-trained  subjects  to 

determine  if  a  general  "good  tracker"  exists  or  if  numerous  repli¬ 
cations  are  needed  to  become  proficient  with  various  types  of 
trajectories. 

After  basic  tracking  performance  has  been  established,  it  is  recommended 
that  additional  research  be  directed  at  experiments  involving  operator 
strategy.  Specific  emphasis  should  be  given  to: 

(1)  Acquisition  and  hand-off  studies  where  tracking,  begins  .it  various 
slant  ranges  within  the  launch  envelope. 

(2)  Operator  strategy  in  a  countermeasure  environment, 

(  1)  Ability  of  operators  to  estimate  launch  envelope  hv  visual  sic. 
and  aspect  of  the  target. 


Appendix  A 

TRAJECTORY  CHARACTERISTICS 


(1)  Plots  should  be  considered  as  approximate  information  only  since 
they  have  been  smoothed  graphically  to  eliminate  noise. 


(2)  Sign  conventions: 


(11)  Note  that  scaling  of  axes  is  not  uniform  across  trajectories. 


I 


TIME  (SEC) 

FLYBY  IB 


FLYBY  IB 


Oas/syaiaw)  aivy  aowy 


(03S/03S/SU313W)  0 


o 

CM 

fO 


TIME  (SEC) 

SLOW  5A 


0.05 


0.136 


o 


(030)  NOI1VA313 


SLOW  5A 


0  024 


o  o  o  o 

!03S/03S/030)  00V  NOI1VA313 


37 


23.8 


(03S/03S/SU313IAI)  31VU  30NVH 


TIME  (SEC) 

SLOW  5A 


140 


CO 

o  2? 

lu  <C 
CO  Q_ 

m  oo 

i=  DC 

LU 


(03Q)  H1DIAIIZV 


'St 


a 

I 


TIME  (SEC) 

ER  S-PASS 


TIME  (SEC) 

ER  S-PASS 


ER  S-PASS 


080 


ER  S-PASS 


o 
c o 


hi 


(wh)  30NVy 


ER  S-PASS 


40U 


TIME  (SEC) 

ER  S-PASS 


(03S/03S/Syai31M)  00V  39NVd 


o 

CD 


CO  CM  i— 

(oaa)  Hinwizv 


70 


TIME  (SEC) 

ER  ZIG-ZAG 


0.60 


(03S/03S/03Q)  00V  HIOIAIIZV 


/Z 


ER  ZIG-ZA 


ER  ZIG-ZAG 


3 


J 


o 


09  0 


ER  ZIG-ZA 


ER  ZIG-ZAG 


FLY 


TIME  (SEC) 

FLYBY  ID 


0.40 


o 

00 


(03S/03S/03Q)  00V  HlfllAIIZV 


o 

LLI 

CO 


I- 


Q 

DO 


I 


81 


P 


FLY 


80  0 


FLY 


(03S/03S/SH3J.3IAI)  00V  30NVd 


APPENDIX  B 

SUBJECT  CONSENT  FORM 


Subject  Consent  for  Participation  In  An  Investigation 
of  Television  Tracking  Performance  as  a  Function  of  Aircraft  Trajectory 

You  are  invited  to  participate  in  a  study  to  determine  the  effects  of 
various  aircraft  trajectory  characteristics  on  television  tracking  perfor¬ 
mance.  We  hope  to  learn  which  characteristics  cause  the  best  performance 
and  which  cause  the  worst.  You  are  invited  to  participate  in  this  study 
because  you  are  a  member  of  the  SRL  subject  pool. 

If  you  decide  to  participate,  you  will  be  instructed  concerning  specific 
details  of  your  task.  You  will  be  tracking  simulated  aircraft  trajectories 
using  either  the  azimuth  or  elevation  handwheel  on  the  AFAMRL  SAM  simulator. 
Four  trajectories  will  be  presented  to  you  during  each  fifty-minute  session. 
You  will  track  for  two  sessions  per  day  for  approximately  six  days.  There 
are  no  known  medical  hazards  associated  with  this  task. 

Any  information  that  is  obtained  with  this  study  and  that  can  be  iden¬ 
tified  with  you  will  remain  confidential  and  will  be  disclosed  only  with 
your  permission.  If  you  give  your  permission  by  signing  this  document,  we 
plan  to  disclose  only  summary  statistical  data  concerning  your  performance, 
with  no  personal  reference  to  you.  You  will  be  paid  your  regular  salary 
while  participating. 

Your  decision  to  participate  will  not  p.  judice  your  future  relations 
with  SRI,.  If  you  decide  to  participate,  you  are  free  to  withdraw  your  con¬ 
sent  and  to  discontinue  participation  at  any  time  without  prejudice. 

If  you  have  any  questions,  we  expect  you  to  ask  us.  If  you  have  any 
additional  questions  later,  Mr.  Evan  Rolek  or  Dr.  Diana  Nelson  (2t)8-'j9bO) 
will  be  happy  to  answer  then. 

YOU  Wild,  BE  GIVEN  A  CORY  OF  THIS  FORM  TO  KEEP. 

YOU  ARE  MAKING  A  DECISION  WHETHER  OR  NOT  TO  PARTICIPATE.  YOUR  SIGNATURE 

INDICATES  THAT  YOU  HAVE  DECIDED  TO  PARTICIPATE  HAVING  READ  THE  INFORMA¬ 
TION  PROVIDED  ABOVE. 
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Appendix  C 

SAM  SIMULATOR  DAILY  CALIBRATION  CHECKLIST 

SAM  SIMULATOR  DAILY  CALIBRATION  CHECKLIST 


1 .  Date  _ 

2.  Time  _ 

3.  Hardware  Condition: 

a.  Elevation  Console  (2nd  fir)  _ 

b.  Azimuth  Console  (2nd  fir)  _ 

c.  Video  Equipment  (3rd  fir)  _ 

4.  SAM  Console  TV  Monitor  Calibration: 

a.  Elevation  Brightness  Levels  (ft-L) 

Sky  (center  of  bkg  film)  _ 

Ground  (center  of  bkg  film)  _ 

Sky  (EL  =  17°,  AZ  =  67°,  darkest  area)  _ 

Aircraft  at  100%  Atmospheric  Transmission 

b.  Azimuth  Brightness  Levels  (ft-L) 

Sky  (center  of  bkg  film)  _ 

Ground  (center  of  bkg  film)  _ 

Sky  (EL  =  17°,  AZ  =  67°,  darkest  area)  _ 
Aircraft  at  100%  Atmospheric  Transmission 

3.  Bores ight /Aircraf t  Alignment  _ 

b.  Tracking  Error  Detection  and  Scoring  _ 

7.  Strip  Chart  Comparison: 
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Appendix  D 

ENSEMBLE  TRACKING  ERROR  PLOTS 


(1)  Condition  number:  means  trajectory  number 

Subject  number:  means  team  number 

Err:  means  error. 


(2)  Plots  based  on  16  replications  for  Team  1  and  20  replications  for 
Team  4. 


(3)  Sign  conventions: 
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Appendix  E 

CELL  MEANS  AND  STANDARD  DEVIATIONS 


CELL  MEANS  OF  RMS  AZIMUTH  SCORES  (mrad) 
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CELL  STANDARD  DEVIATIONS  RMS  AZIMUTH  SCORES  (mrad) 
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CELL  MEANS  OF  RMS  ELEVATION  SCORES  (rarad) 


Trajectory 

FLYBY  IB 

1 

SLOW  5A 

2 

ER  S-PASS 

3 

ER  ZIG-ZAG 

4 

FLYBY  ID 

5 

TEAM 

■ 

.408 

.247 

.688 

.465 

.588 

fl: 

.501 

.284 

.842 

.632 

.622 

CELL  STANDARD  DEVIATIONS  RMS  ELEVATION  SCORES  (mrad) 
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CELL  MEANS  OF  RMS  COMBINED  SCORES  (mrad) 
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Appendix  F 

RESULTS  OF  TUKKY  USD  TESTS 

RESULTS  OF  TUKEY  HSD  TEST:  RMS  AZIMUTH  SCORES 


TRAJECTORIES 

BEING 

COMPARED 


DIFFERENCE 

BETWEEN 

MEANS 
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SIGNIFICANCE 


NS 


NS 


NS 

NS 


Note: 

(1)  NS  Not  Significant 

(2)  Critical  Value:  9  01 . 5.  4  =  9  96 

(3)  Hesults  also  shown  on  Figure  1  2 
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RESULTS  OF  TUKEY  USD  TEST:  RMS  ELEVATION  SCORES 
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